Skip to content

[201811] Fix swsssdk build by installing redis explicitly#7414

Merged
yxieca merged 3 commits intosonic-net:201811from
samaity:build_fix
Apr 29, 2021
Merged

[201811] Fix swsssdk build by installing redis explicitly#7414
yxieca merged 3 commits intosonic-net:201811from
samaity:build_fix

Conversation

@samaity
Copy link
Copy Markdown
Collaborator

@samaity samaity commented Apr 23, 2021 •
edited by yxieca
Loading

Signed-off-by: Sangita Maity samaity@linkedin.com

Why I did it

for 201811 build image, swsssdk wheel package is not getting build due to redis==2.10.6 requirement issue. Notied that after upgrading pip to 20.3.3, we are experiencling this issue.

the issue

21:10:41  /sonic/src/sonic-py-swsssdk /sonic
21:10:41  running test
21:10:41  Searching for redis==2.10.6
21:10:41  Reading https://pypi.python.org/simple/redis/
21:10:41  Couldn't find index page for 'redis' (maybe misspelled?)
21:10:41  Scanning index of all packages (this may take a while)
21:10:41  Reading https://pypi.python.org/simple/
21:10:41  No local packages or download links found for redis==2.10.6
21:10:41  error: Could not find suitable distribution for Requirement.parse('redis==2.10.6')
21:10:41  [  FAIL LOG END  ] [ target/python-wheels/swsssdk-2.0.1-py3-none-any.whl ]
21:10:41  slave.mk:422: recipe for target 'target/python-wheels/swsssdk-2.0.1-py3-none-any.whl' failed
21:10:41  make: *** [target/python-wheels/swsssdk-2.0.1-py3-none-any.whl] Error 1
21:10:43  Makefile.work:132: recipe for target 'target/sonic-aboot-broadcom.swi' failed
21:10:43  make[1]: *** [target/sonic-aboot-broadcom.swi] Error 2
21:10:43  make[1]: Leaving directory '/data/johnar/workspace/broadcom/buildimage-brcm-201811'
21:10:43  Makefile:6: recipe for target 'target/sonic-aboot-broadcom.swi' failed

So, what I have understood till now, if pip v20.3.3 is able to produce a wheel that is not installable because it raises pip._vendor.packaging.version.InvalidVersion or some error like that (resource- > pypa/pip#9206), it just raises an exception when building the wheel.

SO, this issue occurs when we pinned down pip to 20.3.3 in short.

As of now, there are two solutions mentioned below.

  1. pin down pip to 20.3.3(which it is) and explicitly install packages.
  2. pin down pip to 20.3.4(pip wheel now verifies the built wheel contains valid metadata, and can be installed by a subsequent pip install.)(resource-> https://pip.pypa.io/en/stable/news/) -- didn't try yet

How I did it

  • Install nose explicitly for mockredispy
  • Install redis==2.10.6 for swsssdk tests.

How to verify it

Build got successful.

Build Fix
afcfb10 
Signed-off-by: Sangita Maity <samaity@linkedin.com>
@azure-pipelines
Copy link
Copy Markdown

azure-pipelines bot commented Apr 24, 2021

Commenter does not have sufficient privileges for PR 7414 in repo Azure/sonic-buildimage

@lguohan
Copy link
Copy Markdown
Collaborator

lguohan commented Apr 24, 2021

can you add description? why do you do it?

@samaity samaity changed the title Build Fix [201811] Fix swsssdk build by installing redis explicitly Apr 24, 2021
@samaity
Copy link
Copy Markdown
Collaborator Author

samaity commented Apr 24, 2021

can you add description? why do you do it?

Updated the description Guohan.

As of now, there are two solutions as mentioned above.

  1. pin down pip to 20.3.3(which it is) and explicitly install packages. -- built successfully.
  2. pin down pip to 20.3.4(pip wheel now verifies the built wheel contains valid metadata, and can be installed by a subsequent pip install.)(resource-> https://pip.pypa.io/en/stable/news/) -- building locally; still waiting for completion.

@lguohan
Copy link
Copy Markdown
Collaborator

lguohan commented Apr 24, 2021

@yxieca , can you check?

@yxieca
Copy link
Copy Markdown
Contributor

yxieca commented Apr 26, 2021

@samaity my internal 201811 build doesn't seem to need this change.

@samaity
Copy link
Copy Markdown
Collaborator Author

samaity commented Apr 26, 2021

@samaity my internal 201811 build doesn't seem to need this change.
@yxieca, I see build image for 201811 branch is getting failed in upstream too. Kindly let me know if I am missing something.


14:02:05  [ FAIL LOG START ] [ target/python-wheels/swsssdk-2.0.1-py3-none-any.whl ]
14:02:05  /sonic/src/sonic-py-swsssdk /sonic
14:02:05  running test
14:02:05  Searching for redis==2.10.6
14:02:05  Reading https://pypi.python.org/simple/redis/
14:02:05  Couldn't find index page for 'redis' (maybe misspelled?)
14:02:05  Scanning index of all packages (this may take a while)
14:02:05  Reading https://pypi.python.org/simple/
14:02:05  No local packages or download links found for redis==2.10.6
14:02:05  error: Could not find suitable distribution for Requirement.parse('redis==2.10.6')
14:02:05  [  FAIL LOG END  ] [ target/python-wheels/swsssdk-2.0.1-py3-none-any.whl ]
14:02:05  slave.mk:422: recipe for target 'target/python-wheels/swsssdk-2.0.1-py3-none-any.whl' failed
14:02:05  make: *** [target/python-wheels/swsssdk-2.0.1-py3-none-any.whl] Error 1
14:03:27  make[1]: *** [Makefile.work:135: target/sonic-broadcom.bin] Error 2
14:03:27  make[1]: Leaving directory '/data/johnar/workspace/broadcom/buildimage-brcm-201811'
14:03:27  make: *** [Makefile:8: target/sonic-broadcom.bin] Error 2
14:03:27  +++ --- Making target/sonic-aboot-broadcom.swi --- +++
14:03:27  BLDENV=stretch make -f Makefile.work stretch
14:03:27  make[1]: Entering directory '/data/johnar/workspace/broadcom/buildimage-brcm-201811'

@yxieca
Copy link
Copy Markdown
Contributor

yxieca commented Apr 26, 2021

@samaity I guess the problem start to happen on public build too: https://sonic-jenkins.westus2.cloudapp.azure.com/job/broadcom/job/buildimage-brcm-201811/90/console

In that case, I think we should pin the version to 20.3.3 to match with image build change a5b9abc

@zhenggen-xu
Copy link
Copy Markdown
Collaborator

zhenggen-xu commented Apr 27, 2021

@samaity I guess the problem start to happen on public build too: https://sonic-jenkins.westus2.cloudapp.azure.com/job/broadcom/job/buildimage-brcm-201811/90/console

In that case, I think we should pin the version to 20.3.3 to match with image build change a5b9abc

BTW: even with this change, we would hit other issues like the one from 201911:
https://sonic-jenkins.westus2.cloudapp.azure.com/job/broadcom/job/buildimage-brcm-201911/190/console

@yxieca yxieca marked this pull request as ready for review April 29, 2021 05:44
@yxieca
Copy link
Copy Markdown
Contributor

yxieca commented Apr 29, 2021

#7459 will address the snmp docker build issue.

@yxieca
Copy link
Copy Markdown
Contributor

yxieca commented Apr 29, 2021

/azp run

@azure-pipelines
Copy link
Copy Markdown

azure-pipelines bot commented Apr 29, 2021

Azure Pipelines successfully started running 1 pipeline(s).

@yxieca yxieca merged commit b437b77 into sonic-net:201811 Apr 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Build Included in 201811 Branch

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@samaity @lguohan @yxieca @zhenggen-xu